Visually guided grasping in unstructured environments

We present simple and robust algorithms which use uncalibrated stereo vision to enable a robot manipulator to locate, reach and grasp unmodelled objects in unstructured environments. In the rst stage, an operator indicates the object to be grasped by simply pointing at it. Next, the vision system segments the indicated object from the background, and plans a suitable grasp strategy. Finally, the robotic arm reaches out towards the object and executes the grasp. Uncalibrated stereo vision allows the system to continue to operate in the presence of errors in the kinematics of the robot manipulator and unknown changes in the position, orientation and intrinsic parameters of the stereo cameras during operation.